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lntroduct ion 

The  theoretical  results  and  Monte-Carlo  studies  in  the  area  of  robustness 
have  in  the  main  focused  on  symmetric  distributions  (Andrews,  et  al  (1972))  or 
procedures  which  are  not  scale  invariance  (which  effectively  eliminates  most 
problems  due  to  asymmetry  when  the  number  of  dimensions  in  the  problem  is 
fixed).  Recently,  Huber  (1973)  and  Bickel  (1978)  have  examined  situations  in 
which  the  asymmetry  of  errors  can  lead  to  quite  complicated  results.  In  this 
paper  we  study  the  effects  of  asymmetric  errors  in  regression. 

A major  difficulty  with  considering  asymmetric  errors  has  been  that  loca- 
tion (intercept)  is  not  uniquely  defined.  However,  asymmetric  data  do  occur 
and  there  are  situations  where  data  transformation  to  achieve  symmetry  either 
make  no  sense  or  are  not  possible.  In  regression,  it  might  be  conjectured 
that  asymmetry  has  different  effects  on  intercept  and  slope  (see  Section  3); 
if  so,  there  will  be  situations  where  one  might  invest  much  effort  in  data 
transformations,  when  the  parameters  of  interest  are  not  influenced  by  the 
asymmetry. 

Carroll  (1978c)  considered  asymmetric  errors  in  regression  and  illustrated 
his  results  by  means  of  a Monte-Carlo  study,  using  simple  linear  regression 
with  a uniform  design.  These  results  indicate  that  asymmetry  effects  robust 
M-estimates  of  regression  only  through  the  intercept  term,  which  may  be  biased 
and  have  a variance  that  cannot  be  consistently  estimated  by  the  usual  variance 
estimates.  The  purpose  of  this  report  is  to  expand  Carroll's  (1978c)  Monte- 
Carlo  study  to  a wide  variety  of  designs.  The  results  are  almost  staggering 
in  their  consistency  (and  confirm  the  results  of  Carroll  (1978c)),  especially 
in  view  of  the  fact  that  the  designs  we  consider  range  from  balanced  to 
unbalanced  with  a large  amount  of  multicollinearity . 


In  Section  2,  we  review  the  theory  of  M-estimates  as  it  applies  to  situ- 
ations where  the  errors  are  possibly  asymmetric . In  Section  X,  we  report  the 
Monte-Carlo  results,  while  in  Section  4 we  present  our  conclusions. 


M-est imates 

In  the  one  sample  problem,  we  have  a sample  Xj ,X7, . . .,Xn  from  a distribu- 
tion function  F.  Our  aim  is  to  estimate  the  center  of  the  distribution. 

Huber  (1964)  defines  the  center  0 by 


(2.1) 


/ 4»(x  - G)dF(x)  = 0, 


where  ^ is  a skew -symmetric  ty(x)  = -\j»  (-x))  nondecreasing  function.  If 
^(x)  = x,  0 is  the  population  mean.  Huber  (1964)  then  proposed  to  estimate  0 
by  solving 


(2.2) 


/ i)»(x  - 0)dFn(x) 


-1 


l - o) 

i=  l 


0, 


with  the  solution  denoted  by  T^.  If  ^(x)  = x,  we  obtain  the  sample  moan,  which 
is  well-known  to  lack  robustness  against  outliers.  One  possible  choice  of 
to  achieve  this  robustness  is 


^t(x)  = max(-k,  min(x.k)), 

where  in  our  Monte-Carlo  study  wo  take  k = 2.  The  estimate  obtained  by  solving 
equation  (2.2)  is  not  scale  equivariant;  to  achieve  this  property,  Huber  (1977) 
proposes  solving  the  system  of  equations 
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(2.3) 


(2.4) 


n'1  l i|»((X  - T )/s  ) » 0 

i = l 


(n-1)'1  l ^((X.  - Tn)/sn)  = a = E ^(Xj), 
i=l 


where  the  last  expectation  is  taken  under  the  standard  normal  distribution.  If 

P P 

F is  symmetric,  it  can  be  shown  that  if  -*  T(F),  o(F) , then 


nJ(Tn  - T(F) ) - N (0,  A(*,F)). 


where  the  asymptotic  variance  is  (T(F)  = 0,  o(F)  = 1) 


A(<J»,F)  = / «J»2(x)dF(x)/{  / ^'(x)dF(x)}4 


This  suggests  that  the  variance  of  be  estimated  by 


(2.5)  D - s2  (n-1)'1  V <JT((X.  - T )/s  )/{n_1  Y ((X.  - T )/s  ) }2  , 
n n . *•,  i n n ,L.  T t n n 

x*l  i=l 


as  is  suggested  by  Gross  (1976). 

When  F is  asymmetric,  these  results  are  not  true  and  in  fact  Carroll 
(1978a)  (extended  to  multivariate  situations  in  Caroll  (1978b))  shows  the 
following  result. 


Lemma  l_.  Suppose  that  for  some  constants  T(F),  o(F)  we  have  that  T(F), 

n'i(sn  - 0(F))  = 0^(1)  and  F.tp ( ( X j - T(F))/o(F))  = 0.  Taking  T(F)  = 0,  o(F)  = 1 
(without  loss  of  generality),  for  smooth. 


(2.6)  (F.f  ^(Xj))Tn  = n'1  1 iKXj)  ♦ (l-sn)F.pXl4,'(X1)  ♦ 0 (n‘l) 

i=l 
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This  result  says  that  will  not  be  a consistent  estimate  of  the  vari- 

i 

ance  of  if  F is  asymmetric,  Carroll  (1978c)  shows  by  examples  that  there 

exist  situations  for  which 


E D 

j~ — < .65  , 

Var(n  *T  ) 
n 

and  there  are  presumably  distributions  where  this  is  worse.  The  question  we 
want  to  answer  is  how  asymmetry  will  influence  robust  regression  estimates. 

The  model  we  consider  is  (the  use  of  o0  will  become  clear  later) 

(2‘7)  Bq  ♦ eioQ  (i  = 1,  — ,n)  , 

where  the  e.  are  i.i.d.  random  variables  with  Etf»(e,)  = 0 and  x.  = (1  x . x ) 

1 1 — i il  *ip 

We  consider  a version  of  Huber's  Proposal  2 (Huber  (1977),  p.  37),  which  involves 
solving  the  equations 

-1  n 

(2.8)  n l 4>( (y  - x.g  )/s  ) = 0 

i=l 

-1  n 2 

(2-9)  (n-p)  l \j>fc((y.  - x.g  )/s  ) = a. 

i*l  i — i-=n  n 

While  we  will  assume  the  x are  constants,  the  first  two  conditions  of  Lemma  2 
(to  follow)  are  reasonable  and  may  be  justified  by  quoting  results  of  Maronna 
and  Yohai  (1978).  In  a subset  of  their  paper,  they  assume  (ypXj),  (y,,x_,),... 
is  a sample  from  a distribution  function  P with  ^ o,  solving 

EiH(y  - x jf^l/o^)  = 0 
EiT((y  - x j^l/Og)  = a. 
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Defining  = (y\  - x.^  ^o^°0’  ^ey  s^ow  essentially  that  if  the  are  inde- 
pendent of  x^  and  if  fl^,  sn  satisfy  (3.2)  and  (3.3)  (the  latter  with  (n-p) 
replaced  by  n) , then  n (ft^  - 6^)  and  n (sp  - oQ)  are  asymptotically  normally 
distributed . 

The  proof  of  our  result  involves  Taylor  expansions  along  the  lines  of 
Carroll  (1978a,  1978b)  and  is  omitted.  Recall  that  our  £ are  non-stochastic. 

Lemma  2 . Suppose  that 


"'“(fin  - §o>  * Opd) 

"'!(sn  - °0>  ■ V*) 

n ^X'X  -»  V(positive  definite) 

1 11 

n-1  l x.  h-  (1,0,0... .,0)  = W 
i-1  _1 

Then,  for  ip  sufficiently  smooth, 

(2.10)  (a3V  - (a1a4/a2)W'W)(|n  - BqJ/Oq 

= n'1  l (x^ifi  (et)  - (aj/a^W'OJi2^)  - a)} 
i*l 

♦ 0 (n'1). 

where 

aj  = F.  F.jip'CEj)  a3  « Ity'Cej) 

a2  = 2^‘  c]^^r;i^,^eP  a4  = 2K  'MCc j )tp* (C| ) . 


b 


In  simple  linear  regression  with 


1 0 
0 v. 


we  obtain  that  if  31 


(6-  . , 0 , ) and  0' 

int’  pslope'  -!-n 


(0-  » » 0 , ) , then 

xnt  slope  ' 


Corol lary . Under  the  conditions  of  Lemma  3,  for  simple  linear  regression 


(2.11) 


(6int  - *int)AlO 


= (a  - (a.a./a.))  1 n 1 £ {iHt:.)  - (a, /a,)  (t|T (e . ) -a) } ♦ 0 

^ 1 1 1 P 

(2’12)  Aslope  - eslope)/o0  = (a3  V*  xil  *(eiJ  + V"'^- 


Similar  results  hold  for  the  general  regression  problem. 

Lemma  2 says  that,  at  least  theoretically  for  large  n when  the  dimension 
of  the  problem  remains  fixed,  the  effect  of  asymmetry  of  errors  on  robust 
estimators  of  regression  occurs  mainly  in  the  intercept.  If  F is  symmetric, 
Lemma  2 says  that  since  = 0, 


Var(nJ(0n  - 0)/oo) 

. ef»2<V  v-i 

{Ep  ifi1  (Xj)  }2 

and  it  has  been  suggested  (Gross  (1977))  that  we  estimate  this  variance  by 
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(2.13) 


> 


where  r.  * (v.  - x.  8 )/s  . What  is  clear  from  Lenusa  2 and  the  Corollary 
i i — l n n 

are  the  following. 


1 


Theoret ical  Conclusions 

(i)  The  effect  of  asymmetry  of  errors  on  robust  regression  estimates  is 
evidenced  only  in  the  intercept,  which  will  tend  to  be  "biased"  and  have  a 
variance  which  is  larger  than  expected  from  the  symmetric  case. 

(ii)  will  be  a consistent  estimate  of  the  variance  of  the  slopes,  but 
will  be  generally  inconsistent  for  the  variance  of  the  intercept. 

(iii)  will  be  a consistent  estimate  of  the  covariance. 


In  the  next  section  we  illustrate 
study . 

Mont e- Carl o Experi ment 

Let  Z be  a standard  normal  random 
presented  here  are  as  follows: 

IZE£ 

L 

z ♦ . ioz2 

z ♦ . 50Z2 

Negative  Exponential,  mean  1.25 
.50  Exp(Z) 


the  results  with  a large  Monte-Carlo 


variable.  The  five  distributions 


Code 


lhe  stcond  (.102“)  is  only  slightly  skewed  and  was  chosen  to  be  reasonably  repre- 
sentative of  the  class  of  distributions  close  to,  but  not  exactly,  symmetric.  Such 
data  might  arise  for  example  from  data  transformations  which  only  to  achieve  approx- 
imate symmetry.  If  N is  the  number  of  Monte-Carlo  experiments  and  Y,,...,Y  the 

1 V 

realized  value  of  an  estimate,  the  .Monte-Carlo  variance  is  defined  to  be 


(3.1) 


» H - v: 


All  the  models  are  of  the  form 

Y = X 6 ♦ e , 

where  B'  = (By,  Bj,  ....  6p) . Let  ^,...,6^  denote  the  N realized  values  of 
the  robust  estimate  of  g,  with 

V ■ <«io V- 

form  the  diagonal  matrix  W'  with  j^h  diagonal  clement 


» X % - X h/  • 

Then,  in  all  our  tables,  E(g^)  denotes  the  average  value  of  the  g. . , i.e., 

E V * j,  «,j  • 

The  term  V((y  is  the  standardized  Monte-Carlo  variance  of  the  robust  estimate 
of  gj  and  is  the  ith  diagonal  element  of  the  matrix 


X'XW. 


2 1 2 

Under  symmetry,  V(B^)  ~ E ip  (Cj)/{E  ip  (e^) } , so  that  the  usual  estimate  of 

V(B-)  (Gross  (1977))  is  denoted  by  V(B.)  and  is  defined  by 
J 2 J 

s„  n o 

. * J,  * ^ 

V(V  = , ‘ lf  ■ 

n J,  * <ri>> 

1 = 1 

Hence,  the  tables  contain  the  following  information  (NITER  = number  of 
iterations) . 

(i)  E Bj  = Average  value  of  the  robust  estimate  obtained  in  the  study, 

(ii)  V(Bj)  = The  Monte-Carlo  variance  of  the  estimate  of  Bj  • 

To  obtain  the  "true"  Monte-Carlo  variance  of  , merely  multiply  V(Bj)  by  the 


diagonal  element  of  (X'X)  */N. 

(iii)  V(3j)  = The  usual  estimate  of  V(B^)  obtained  assuming  symmetry. 

(iv)  Ratio  = V(B^)/V(Bj)  = Ratio  of  estimated  variance  to  true  variance. 
The  following  designs  were  considered  in  our  Monte-Carlo  experiment.  In 
all  cases,  the  number  of  iterations  is  1200. 


Design  #1.  Here  we  have 


Y.  .,  = y + a.  + y . + e.  , where 

l jk  M i 'j  ljk 


i = 1,2 


j = 1,2,3 


k = 1, 2,3,4 


(Hence  n = 24) 


«i  + (*2  = 0 


Yl  + y2  + y3  = 0. 
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This  is  a balanced  2x3  design.  In  Table  1 we  call  Bq  = y,  g * g 

= Yl*  £3  = Y2*  1116  true  values  used  are  BQ  = 2.50,  Bj  = -.50,  B2  = -1.00, 

b3  ■ 0.00. 


Design  #2.  Here  we  have  simple  linear  regression 


Y. 

l 


Vi 


+ 


e. 

i 


(i  = 1, . . . ,20) 


with  ^0  = 1-00»  = -50  and  the  values  of  the  being  -.95,  -.90, . . . , .90, .95. 

Design  #3.  This  is  the  same  as  Design  #2  except  the  x's  are  unbalanced  and 
given  by 

x 

-0.349 

-0.344 

-0.333 

-0.318 

-0.297 

-0.270 

-0.239 

-0.202 

-0.160 

-0.113 

-0.060 

-0.002 

0.060 

0.128 

0.202 

0.281 

0.365 

0.454 

0.549 

0.649 
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Design  »5.  This  is  a regression 

Yt  = 1.00  ♦ .50Xn  ♦ .25X.,  ♦ e^, 

i 

where  the  X._  arc  essentially  translates  of  X...  There  are  n=20  design  points 


arrayed  in  a uniform 


manner  with  X'X 

being  a diagonal 

in 

il2 

-0.34435 

-0.21371 

-0.34205 

-0.21370 

-0.33745 

-0.21366 

-0.33056 

-0.21351 

-0.32136 

-0.21317 

-0.30987 

-0.21250 

-0.29608 

-0.21135 

-0.28000 

-0.20953 

-0.26161 

-0.20682 

-0.2409.3 

-0.20296 

-0.21795 

-0.19766 

-0.19266 

-0.19062 

-0.16509 

-0.18147 

-0.13521 

-0.16985 

-0.10304 

-0.15534 

-0.06856 

-0.13750 

-0.03179 

-0.11584 

0.00728 

-0.08987 

0.04865 

-0.05904 

0.09231 

-0.02279 

0.13827 

0.01949 

0.18654 

0.06843 

0.23710 

0.12470 

0 . 28995 

0.18899 

0.34511 

0.26204 

0.40257 

0 . 34460 

0.46232 

0.43746 

0.52437 

0.54146 

0.58872 

0.65744 

0.65537 

0.78629 

x. 


$ 
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Design  »6. 
design . 


This  is  the  same  model  as  in  Design  #5  but  with  a highly  unbalanced 


-0.96667 
-0.90000 
-0.83333 
-0.76667 
-0.70000 
-0.63333 
-0.56667 
-0.50000 
-0.43333 
-0.36667 
-0.30000 
-0.23333 
-0.16667 
-0.10000 
-0.03333 
0.03333 
0.10000 
0.16667 
0.23333 
0 . 30000 
0 . 36667 
0.43333 
0.50000 
0.56667 
0.63333 
0.70000 
0.76667 
0.83333 
0.90000 
0.96667 


0.60148 

0.47704 

0.36148 

0.25481 

0.15704 

0.06815 

-0.01185 

-0.08296 

-0.14519 

-0.19852 

-0.24296 

-0.27852 

-0.30519 

-0.32296 

-0.33185 

-0.33185 

-0.32296 

-0.30519 

-0.27852 

-0.24296 

-0.19852 

-0.14519 

-0.08296 

-0.01185 

0.06815 

0.15704 

0.25481 

0.36148 

0.47704 

0.60148 
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Design  #1 


The 

true  values 

are  =*  2.50, 

* -.50, 

g2  = -1.00, 

e3  = 0.00 

Z 

.10Z2 

.soz2 

NE 

EXP(Z) 

E eo 

2.50 

2.49 

2.44 

2.45 

2.41 

V(60) 

.99 

.98 

1.46 

1.49 

.71 

< 

/— s 

O 

w 

.99 

.99 

1.17 

1.27 

.52 

Ratio 

1.00 

1.01 

.80 

.85 

.72 

E 

- .50 

- .50 

- .50 

- .51 

- .50 

V(Bj) 

.98 

.99 

1.16 

1.38 

.51 

V(3j) 

.99 

.99 

1.17 

1.27 

.52 

Ratio 

1.01 

1.00 

1.01 

.93 

1.01 

e e2 

-1.00 

-1.00 

-1.00 

O 

O 

H 

1 

-1.00 

VC32) 

.96 

.97 

1.12 

1.33 

.50 

V(32J 

.99 

.99 

1.17 

1.27 

.52 

Ratio 

1.03 

1.02 

1.05 

.96 

1.03 

E s 

.01 

.00 

.01 

-.02 

.00 

V(33) 

1.04 

1.03 

1.18 

1.30 

.53 

VC83) 

.99 

.99 

1.17 

1.27 

.52 

Ratio 

.95 

.96 

.99 

.98 

.97 

L 


I 


The  true  values  are  6q  3 1.00,  Bj  = .50. 


Z 

2 

• 10Z 

.50Z2 

NE 

EXP(Z) 

E eo 

1.00 

.99 

.92 

.94 

.90 

V(B0) 

1.05 

1.06 

1.49 

1.50 

.70 

V(60) 

1.00 

.99 

1.11 

1.24 

.48 

Ratio 

.95 

.94 

.75 

00 

CsJ 

.69 

E Bj 

.49 

.49 

.50 

.48 

.50 

VCBj) 

1.03 

1.02 

1.17 

1.24 

.53 

VCBj) 

1.00 

.99 

1.11 

1.24 

.48 

Ratio 

.97 

.98 

.95 

1.00 

.92 

IK 


V(S.) 

V(^) 


Ratio 


.50 

1.00 

1.00 

1.00 


.51 

1.06 

1.06 

1.00 


.51 

.43 

.43 

1.00 


Design  »5 


The  true 

values  are 

e0  = i.oo,  gj  « 

.50, 

2 

2 

Z 

.102“ 

.502“ 

1 

F.  gQ 

1.00 

.98 

.91 

v(V 

.99 

1 .01 

1.47 

1 

1.00 

1.00 

1.02 

1 

Ratio 

1.02 

.99 

.74 

E Bl 

.49 

.49 

.50 

Vfgj) 

1.02 

1.02 

1.14 

1 

V(3X) 

1.00 

1.00 

1.09 

1 

Ratio 

.99 

.98 

.96 

E B2 

.24 

.25 

.27 

vcs,) 

1.00 

1.00 

1.12 

1 

^(S2) 

1.00 

1.00 

1.09 

1 

Ratio 

1.00 

1.00 

.97 

1 

X’X  » 

f30  0 

0 

Corr.  Matrix  = (l 

0 

0 

9.99 

0 

0 

1 

0 

2.65 

0 

0 

1 
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Design  *6 


The  true 

values  are 

B0  =■  1.00,  ^ = 

.50,  B,  « 

.25. 

Z 

? 

.102“ 

2 

.502* 

NE 

EXP (2) 

E *o 

1.00 

.98 

.91 

.92 

.89 

V<V 

.99 

1.01 

1.47 

1.51 

.67 

V(^0) 

1.00 

1.00 

1.09 

1.18 

.46 

Ratio 

1.02 

.99 

.74 

.79 

.68 

E *1 

.49 

.49 

.46 

.48 

.46 

V(Bj) 

.98 

.97 

1.08 

1.16 

.45 

^(Bj) 

1.00 

1 .00 

1.09 

1.18 

.46 

Rat  io 

1.02 

1.02 

1.09 

1.03 

1.02 

E B 

.24 

.26 

.31 

.30 

.30 

v(b2) 

.97 

.97 

1.10 

1.17 

.46 

v'(b2) 

1.00 

1.00 

1.09 

1.18 

.46 

Rat  io 

1.03 

1.02 

.99 

1.01 

1.00 

J1 


Conclusions 

The  Monte-Carlo  results  are  surprisingly  consistent.  It  appears  that,  in 
robust  regression,  one  can  accurately  estimate  slopes  and  their  variances,  even 
if  the  design  is  highly  unbalanced  with  considerable  multicol linearity ; the 
conclusion  holds  over  a wide  class  of  distributions  varying  from  the  normal  to 
a heavy  tailed,  very  skewed  distribution  (EXP(Z)).  However,  estimating  inter- 
cept (and  especially  its  variance)  is  complex  if  the  distributions  arc  heavily 
skewed.  As  in  Carroll  (1978c),  we  recommend  that  if  one  has  an  unbalanced 
design,  a heavily  skewed  error  distribution,  and  wishes  to  estimate  terms 
involving  the  intercept,  variance  could  be  assessed  by  using  the  weighted 
jackknife  of  Hinkley  (1977). 
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